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Human  Factors  in  Training 
and  Operational  Effectiveness 


FOREWORD 


The  Syster.s  Research  Laboratory  (SRL)  of  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
supports  the  Army's  MANPRINT  (Manpower  and  Personnel  Integration) 
initiative  in  the  testing  and  development  of  new  systems  and  new 
MANPRINT  methods  such  as  HARDMAN  II  (Hardware  vs.  Manpower) 
analysis. 

This  report  discusses  the  joint  effort  of  two  SRL  organiza¬ 
tions,  the  Fort  Hood  Field  Unit  and  the  Manned  Systems  Group,  to 
identify  and  resolve  major  manpower,  personnel,  and  training 
(MPT)  difficulties  that  reduced  the  effectiveness  of  the  Army's 
Remotely  Piloted  Vehicle,  Aquila.  The  effort  was  undertaken  in 
response  to  requests  from  Headquarters,  Training  and  Doctrine 
Command  (TRADOC) ,  and  the  U.S.  Army  Field  Artillery  School 
(USAFAS)  in  August  1987.  In  June  1988,  a  memorandum,  and  a  work¬ 
ing  paper  describing  the  work  reported  herein  were  sent  to  USAFAS 
(the  original  proponent  for  Aquila) ,  the  program  manager,  the 
TRADGC  systems  manager,  and  the  U.S.  Army  Intelligence  Center  and 
School  (the  proponent  for  the  Unmanned  Aerial  Vehicle  (UAV) 
program) . 


Til  i  to  iepOit  eiaphaaittia  1  O  to  to  O  i'i  leaiiieu  fiToiu  this  KAN  PRINT 


intervention  that  may  be  applied  to  4  he  successor  of  Aquila,  the 
UAV.  By  applying  lessons  in  this  way,  key  MPT  issues  and  con¬ 
cerns  can  be  addressed  early  in  the  acquisition  process  and  per¬ 
formance  cf  future  systems  enhanced. 


The  results  of  this  project  were  presented  at  the  30th  An¬ 
nual  Conference  of  the  Military  Testing  Association  in  November 
1988.  Although  Aquila  was  cancelled  in  early  1988,  the  lessons 
learned  from  this  ARI  effort  were  incorporated  into  the  System 
MANPRINT  Management  Plan  for  the  UAV. 
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Or  AQU I  !  TV ,  THE  AR'.Y'S  REMOTELY  PILOTED  VEHICLE: 


MAN  HR TNT  SCITCCT 
LESSONS  LFARNi  D 


EXECUTIVE  SUMMARY 


Poqu i  rer.env  : 


Many  manpower,  personnel,  and  training  (MPT)  problems  were 
discovered  during  testing  and  development  of  Aquila,  the  Army's 
Remotely  Pile: ed  Vehicle  (RPV) .  A  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences  (AP.I)  Systems  Researc* 
Laboratory  ( SRL)  manpower  and  personnel  integration  (KANPRINT) 
Task  Force,  coordinated  by  the  Fort  Hood  Field  Unit,  was  ;  ~ked 
bv  Headquarters  Training  and  Doctrine  Command  and  the  U.S.  A  -my 
Field  Artillery  School  t.o  help  resolve  these  MPT  problems. 


Procedure : 


The  SRL  Task  Force  conducted  an  in-depth  evaluation  of  the 
major  MPT  issues  relating  to  operation  and  maintenance  of  the 
REV.  Among  there  issues  were  an  analysis  of  specific  human  per- 
i cr~an.ee  prosier  •  uncovered  during  Operational  Testing  i  l  (or 
II),  an  extensiv.  review  of  tlie  literature  on  imaging  systems., 
and  a  sensitivity  analysis  applied  to  the  results  ''f  the  original 
hardware  vs.  manpower  (HARDMAN)  analysis  to  incorporate  lessons 
learned  from.  OT  II. 


F  i  nd  i  p.gs : 

1  le  SRL  Task  Force  was  able  to  identify  major  KPT  problems 
and  to  suggest  concrete  ways  in  which  their  impact  on  total  sys¬ 
tem  performance  could  be  minimized.  Most  of  the  recommendations 
suggested  MPT  so1  rations  to  the  problems. 


Utilization  of  Findings: 

Although  the  Aquila  was  cancelled  in  early  1983,  the  lessen? 
learned  from,  this  KAN  PR  I  NT  intervention  have  provided  valuable 
insights  into  the  MPT  issues  that  must  be  addressed  and  resolved 
before  a  now  system  is  built.  Much  of  this  guidance  is  being 
incorporated  into  the  System  MANPRINT  Management  Plan  for  the 
Unmanned  Aerial  Vehicle,  a  family  of  systems  technologically 
similar  to  Aquila. 
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Sl'El'ORT  OF  AQUILA.  THE  ARMY'S  REMOTELY  PILOTED 
VEH  i  :  LESSORS  LEARNED 


introduction 


Background 

Aquila,  the  Army's  Remotely  Piloted  Vehicle  (RPV) ,  is  a 
relatively  small  unmanned  aircraft  that  is  remotely  controlled 
from  a  ground  station  and  is  designed  to  carry  a  payload  that  can 
serve  target  acquisition,  designation,  reconnaissance  and 
surveillance  functions.  Development  of  the  system  can  be  traced 
back  to  1971  when  the  Defense  :  cience  Board  recommended  that  the 
Army  establish  a  program  which  would  apply  mini-RPV  technology  to 
the  firt'  support  functions  of  target  designation  and  adjustment 
of  artillery  fire.  The  concept  of  using  pilotless  aerial 
vehicles  for  military  purposes  was  investigated  as  early  as  World 
War  I,  albeit  with  little  success  (Joint  Electronic  Warfare 
Center,  1986).  It  was  looked  at  again  in  World  War  II,  and  has 
been  under  some  degree  of  research  and  development  ever  since. 
Much  of  the  early  success  was  with  fairly  large  target  drones 
which  simulated  flying  aircraft  and  were  used  for  gunnery 
exercises.  Eventually,  the  idea  of  modifying  such  drones  by 
hanging  cameras  on  them  emerged,  and  this  combined  with  evolving 
technology  created  many  possibilities  for  military  application. 
For  example,  the  development  of  the  electronic  digital  computer, 
progressive  miniaturization  of  components,  and  other  related 
developments  in  the  1960s  and  1970s,  provided  the  possibilities 
for  building  relatively  small  ail  vehicles  which  could 
continuously  send  real  time  digitized  data  to  a  ground  station. 

In  addition,  it  allowed  controllers  to  control  actively  the 
flight  path  or  permit  the  air  vehicle  to  follow  a  preprogrammed 
flight  path  of  relatively  long  duration.  Finally,  such  small 
vehicles  could  be  easily  launched  and  recovered,  and  for  some 
requirements  procured  cheaply  enough  to  be  expendable. 

It  was  in  this  technological  milieu,  combined  with  an 
increasing  concentration  of  Soviet  ground-based  sir  defense 
weapons  posing  a  high  threat  to  piloted  reconnaissance  aircraft, 
that  the  Defense  Science  Board  made  its  recommendation  in  1971  to 
develop  mini-RPVs  for  reconnaissance  and  target  acquisition.  In 
September,  1974,  the  program  formally  got  underway  when  the  U.S. 
Army  Training  and  Doctrine  Command  (TRADOC)  and  the  U.S.  Army 
Materiel  Command  signed  a  letter  of  agreement  to  develop  an  RPV 
system  that  could  demonstrate  how  RPVs  could  be  used  by  ground 
commanders  in  reconnaissance,  target  acquisition,  laser 
designation  of  targets,  and  adjustment  of  artillery  fire.  This 
effort  evolved  into  the  development  of  a  prototype  system  which 
came  to  be  known  as  Aquila  and  underwent  three  series  of  tests: 
Development  Test  (DT)  II  in  1985-1986,  (Cozbv,  1986) ,  Operational 
Test  (OT)  II  in  1986-1987  [U.S.  Army  Operational  Test  and 
Evaluation  Agency  (OTEA) ,  1987],  and  Force  Development  Test  and 
Experimentation  (FDTE)  in  late  1987  [U.S.  Army  TRADOC  Test  and 
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Experj  lentation  Command  (TEXCOM) ,  1988].  Following  the 

conclusion  o  the  FDTE,  the  Army  decided  to  halt  further 
development  and  cancelled  acquisition  plans  for  Aquila.  Th  -* 
Army's  RPV  efforts  have  since  been  combined  with  those  of  the 
other  military  services  under  a  Joint  Program  Office  for  Unmanned 
Aerial  Vehicles  (UAV) ,  and  the  few  Aquila  systems  that  were 
initially  built  for  test  and  evaluation  purposes  are  cux'rently 
serving  as  a  test  bed  for  the  joint  program. 


MANPRINT 

Manpower  and  personnel  integration  (MANPRINT)  is  a  new  Army 
initiative  whose  goal  is  to  optimize  total  system  performance  by 
considering  the  soldier  as  an  integral  part  of  the  system.  In 
order  to  accomplish  this  goal,  issues  and  concerns  relating  to 
manpower,  personnel,  training,  human  factors  engineering,  health 
hazards  and  system  safety  must  be  addressed  early  in  the 
development  of  a  system.  In  this  way  the  potential  adverse 
impact  of  these  factors  on  soldier  and  system  performance  can  be 
minimized. 

The  Army  Research  Institute  (ARI)  Systems  Research 
Laboratory  (SRL)  supports  the  goals  of  MANPRINT  in  the  testing, 
development,  and  acquisition  of  Army  systems.  This  involves  not 
only  support  of  field  testing  but  also  the  development  and  pilot¬ 
testing  of  analytical  MANPRINT  methods  for  potential  users.  The 
current  project  concerns  itself  with  MANPRINT  support  to  TRADOC 
provided  bv  the  SRL  Aquila  MANPRINT  Task  Force.  The  ARI  Fort 
Hood  Field  Unit  was  responsible  for  technical  direction  of  the 
current  project,  and  had  previously  provided  MANPRINT  support  or 
OT  II.  The  ARI  Manned  Systems  Group  was  asked  to  provide 
technical  assistance  in  addressing  MANPRINT  concerns  pertinent  to 
Aquila' s  imaging  system,  and  to  demonstrate  how  procedures  for 
estimating  maintenance  manpower  requirements  could  l:e  improved. 
The  ARI  Fort  Bliss  Field  Unit  provided  technical  support  in 
addressing  cognitive  workload  issues. 

Lessons  learned  from  this  intervention  were  incorporated 
into  recommendations  of  manpower,  personnel  and  training  actions 
that  should  serve  partially  to  ameliorate  some  of  the  problems 
.hat  had  threatened  t o  diminisn  the  performance  of  Aquila  and 
those  unmanned  aerial  systems  that  will  succeed  it. 

The  extensive  testing  and  evaluation  that  Aquila  underwent 
during  its  multiyear  development  has  provided  the  SRL  MANPRINT 
Task  Force  with  a  great  deal  of  useful  information  about  how 
unmanned  aerial  systems  snould  be  designed,  operated  and 
maintained.  The  remainder  of  this  report  will  focus  on  some  of 
the  MANPRINT  problems  and  issues  that  were  brought  to  light 
during  this  period.  The  first  section  will  discuss  three  topics 
pertinent  to  operation  of  the  system:  (1)  Target  Acquisition, 

(2)  Mission  Planning,  and  (3)  Manpower  and  Personnel  Selection. 
The  second  will  demonstrate  how  a  representative  MANPRINT  method 
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developed  by  MU  can  be  employed  as  a  useful  tool  for  estimating 
maintenance  manpower  requirements.  The  results  of  the  cognitive 
workload  research  will  appear  in  a  separate  ARI  report. 


Operator-Related  MANPRINT  Concerns 


Target.  Acquisition 

One  of  the  major  findings  of  OT  II  was  that  Aquila  crews  too 
often  failed  to  detect  targets.  Examination  of  the  results  from 
OT  II  (OTEA,  1987)  indicates  that  overall  detection  rate  was 
around  17%  (24%  of  moving  targets  and  13%  of  stationary  targets) . 
Several  uncontrollable  extraneous  factors  contributed  to  this  low 
detection  rate.  In  addition,  Aquila  crews  were  required  to 
search  too  large  an  area  during  the  test.  The  mean  area 
reconnoitered  during  OT  II  was  17.6  km2,  which  exceeded  the  16 
km2  area  the  Army  considered  to  be  the  maximum  for  a  typical 
three  hour  Aquila  sortie  (U.S.  Army  Field  Artillery  School, 
undated) .  It  also  became  obvious  during  the  test  that  searching 
for  targets  from  an  altitude  of  1500  m  with  a  20°  field  of  view 
is  very  much  like  looking  at  the  world  through  a  straw,  and  is  a 
very  difiicult  and  tedious  process. 

Between  the  completion  of  OT  II  and  the  beginning  of  the 
FDTE,  the  An  /  reevaluated  Aquila  search  requirements  using  data 
from  OT  II  and  experience  learned  from  other  systems  and 
concluded  that  it  was  not  feasible  to  search  so  large  an  area  for 
targets  in  general.  Instead,  it  seemed  better  to  use  other 
intelligence  sources  to  indicate  approximately  where  targets  may 
be,  and  tc  use  such  information  to  cue  Aquila  crews  to  search  a 
smaller  area. 

This  change  was  incorporated  into  the  operating  procedures 
established  for  the  FDTE.  In  addition,  software  was  developed  to 
assist  the  operator  in  performing  reconnaissance  (TEXCOM,  1988). 

A  bookkeeping  function  was  developed  which  automatically  kept 
track  of  the  area  that  had  been  searched.  During  OT  II  operators 
easily  lost  track  of  where  they  were  when  they  took  manual 
control  of  the  system,  thus  making  it  difficult  to  search  an  area 
systematically.  With  the  new  bookkeeping  function  an  operator 
could  take  manual  control  of  the  system,  leave  the  area,  perform 
an  emergency  search  somewhere  else,  and,  providing  he  did  not 
enter  a  new  waypoint,  come  back  and  automatically  resume  the 
search . 

The  other  software  improvement  was  an  automatic  search 
routine  known  as  the  step-stare  technique.  This  technique 
focused  on  a  given  portion  of  the  search  area  for  a  fixed  amount 
of  time,  depending  on  the  amount  of  clutter  in  the  area,  then 
automatically  moved  to  and  focused  on  an  adjacent  area  with  about 
a  10%  overlap,  and  thus,  provided  for  the  systematic  search  of  a 
whole  area.  This  change  increased  the  time  required  to  search  an 


3 


area,  however,  from  12  to  3b  minutes  per  Km2.  Consequently,  the 
Army  reduced  its  maximum  area  search  requirement,  to  6  km2  per 
sortie  (U.S.  Army  Field  Artillery  School,  1987). 

All  of  the  above  changes  were  incorporated  into  the  Aquila 
system  prior  to  the  FDTE  with  the  intention  of  increasing  the 
target  detection  rate.  The  results  of  the  FDTF.  were  encouraging. 
Ninety-eight  percent  of  all  moving  target  airays  and  94%  of  all 
stationary  arrays  that  were  not  camouflaged  were  acquired  by  the 
Aquila  operators  during  the  FDTE  (TEXCOM  1988).  However,  no 
target  arrays  that  were  camouflaged  were  acquired.  Thus,  the 
results  showed  a  definite  ability  to  detect  moving  and 
stationary,  uncamouflaged  targets,  and  an  inability  to  detect 
targets  camouflaged  with  either  artificial  or  natural  material. 


There  were  additional  target  acquisit 
been  identified  during  the  OT  1  ,  and  stil 

during  the  FDTE.  For  example,  operators  f 
to  detect  targets  that  were  in  shadows,  as 
camouflages  a  vehicle  by  parking  it  under 
Aaron  Hyman  (1987),  recently  of  ARI ,  ident 
contributor  to  this  problem,  along  with  a 
Although  the  camera  in  Aquila  was  of  very 
circuitry  associated  with  the  automatic  ga 
automatic  level  control  was  designed  to  cl 
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ip  the  upper  and  lower 
ocathode.  A  linear 
transfer  function  described  the  relationship  between  the  middle 
80%  of  the  photocathode  illumination  and  the  output  signal  that 
was  transmitted  to  the  ground  control  station  and  translated  into 
various  shades  of  grey  on  the  video  display.  when  an  Aquila 
operator  was  searching  a  scene,  the  camera  typically  received  a 
wide  range  of  input  luminances,  the  lower  of  which  often  came 
from,  dark  or  shadowy  areas.  Since  the  lower  10%  of  the  input 
luminances  was  clipped,  the  information  about  objects  in  the 
shadows  was  not  transmitted  to  the  video  monitors  on  the  ground. 
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Dr.  Hyman  suggested  a  number  of  ways  to  solve  this  problem. 

A  procedural  solution  would  be  to  use  a  narrow  field  of  view 
(FOV) ,  such  as  2.7°,  to  search  shadowy  areas  so  that  the  dynamic 
range  of  input  luminance  spans  just  the  shadowy  area.  However, 
this  is  only  feasible  for  short  periods  of  time.  Of  the  three 
Aquila  FOV  options,  2.7°,  7.2°,  and  20°,  operators  tended  to  use 
the  7.2°  and  20°  FOV ’ s  much  more  than  the  2.7°  FOV  because  of  the 
tremendous  decrease  in  the  amount  of  terrain  they  could  observe 
through  a  narrow  2.7°  FOV,  and  the  corresponding  increase  in  the 
amount  of  time  needed  to  search  an  area.  An  engineering 
solution  to  this  problem  would  be  to  provid  the  operator  with 
the  option  of  switching,  as  circumstances  dictate,  to  circuitry 
which  would  not  clip  the  lower  input  luminances  and  would  employ 
a  non-linear  transfer  characteristic  with  a  high  gain  at  the 
lower  end  of  the  input  luminance  distribution.  This  could  be 
used  to  amplify  and  detect  any  signals  that  existed  in  shadows. 
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Forward  Looking  Infrarod  ( FL1R)  cameras  are  seen  by  many 
observers  as  another  solution  to  this  problem  of  detection  of 
targets  in  shadows.  However,  it  should  be  noted  that  FLIR,  while 
having  many  advantages,  does  not  always  produce  as  clear  a 
picture  as  one  would  like.  When  contrast  between  two  objects  is 
low,  it  is  still  difficult  to  distinguish  them,  and  the  Army 
currently  plans  to  provide  for  both  daylight  TV  and  FLIR  as 
potential  payloads  on  its  UAV  system  that  is  in  the  initial 
stages  of  development.  Thus,  the  problem  of  detecting  targets  in 
shadows  with  daylight  TV  cameras  is  one  which  must  be  addressed. 

Another  aspect  of  the  problem  of  detecting  camouflaged 
targets  concerns  the  combination  of  slant  range  and  FOV  for 
searching  an  area.  Dr.  Hyman's  communication  with  various 
experts  in  the  field  of  imagery  analysis  indicated  that  the 
acquisition  of  targets  in  clutter,  such  as  vegetation,  is  a 
somewhat  different  process  than  ir>  open  terrain.  In  the  latter 
situation  an  observer  typically  detects  something,  and  then  after 
obtaining  better  resolution  by  moving  closer  to  it  or  magnifying 
it,  recognizes  it  as  a  particular  type  of  object.  In  clutter, 
however,  it  appears  that  detection  of  an  object  or  target  does 
not  occur  until  resolution  is  sufficient  to  recognize  it. 

Previous  research  (Johnson,  1958)  has  shown  that  recognition  of 
military  vehicles  with  a  probability  of  0.5  requires  about  eight 
(TV)  lines  of  resolution.  Thus  in  clutter,  it  appears  that  about 
eight  linps  of  resolution  are  needed  to  detect  a  target.  Dr. 
Hyman  calculated  that  with  Aquila's  7.2°  FOV,  a  slant  range  of 
1.66  km  was  required  in  order  to  obtain  enemy  target 
representation  (e.g. ,  a  tank)  of  eight  lines  of  resolution. 
However,  Aquila  doctrine  suggested  a  2.5  km  slant  range  for  area 
search  with  the  7.2°  FOV.  Thus,  it  is  possible  that  some  targets 
in  clutter  were  not  detected  during  testing  of  Aquila  because 
adequate  target  resolution  was  not  obtained  at  the  video  monitor 
due  to  the  non-optimal  combination  of  slant  range  and  FOV. 

One  way  to  resolve  this  problem  would  be  to  use  a  narrower 
FOV.  This  would  give  more  lines  of  resolution  to  a  target  of 
given  size.  Another  alternative  would  be  to  use  a  shorter  slant 
range,  such  as  the  1.66  km  slant  range  mentioned  above,  although 
this  would  require  flying  below  that  altitude  which  had  been 
determined  as  the  minimum  for  sustained  survivability. 


Mission  Planning 

Another  problem  uncovered  during  the  OT  II  was  that  Aquila 
crews  accepted  from  higher  headquarters  mission  requests  that 
exceeded  the  capabilities  of  the  system  (TEXCOM,  1988) .  This 
typically  took  the  form  of  crews  accepting  a  mission  to  search  an 
area  larger  than  Aquila  could  reasonably  handle  during  a  three 
hour  sortie.  The  exact  cause  for  this  was  not  empirically 
determined,  but  it  was  thouqht  to  be  due  to  several  factors,  to 
include  lack  of  understanding  of  Aquila's  limitations  by  the 
crews,  .id  a  tendency  to  be  unquestioning  when  receiving  mission 
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orders  from  a  higher  headquarters.  As  a  result,  the  training 
program  was  modified  prior  to  the  FDTE  to  give  Aquila  crews 
training  on  system  limitations  and  the  need  to  resolve  conflicts 
with  higher  headquarters  ever  nonexecutable  mission  requests. 
During  the  FDTE  crews  were  given  ten  incomplete  mission  orders 
and  nine  nonexecutable  mission  orders.  They  were  scored  on 
whether  they  resolved  those  conflicts.  Scoring  occurred  in  two 
ways.  First,  a  subject  matter  expert  (SME) ,  a  senior  warrant 
officer  observed  the  crews  during  the  three  phases  of  a  mission 
(i.e.,  receiving  the  mission  order  at  higher  headquarters, 
planning  the  mission  in  the  Ground  Control  Station,  and  executing 
the  mission  by  flying  the  air  vehicle  and  searching  for  targets) 
and  judged  whether  or  not  they  resolved  mission  request  conflicts 
with  higher  headquarters  (TEXCOM,  1988).  Second,  videotapes  of 
crew  actions  during  all  three  phases  were  scored  by  independent 
evaluators  to  determine  the  frequency  and  distribution  of  crew 
actions  involving:  a)  requests  for  clarification  of  missions 
from  higher  headquarters,  and  b)  explaining  system  capabilities 
to  higher  headquarters  (Nicholson,  Deignan,  &  Smootz ,  1988). 

The  results  were  conclusive.  The  SME  determined  that  crews 
identified  and  adequately  resolved  all  19  conflicts  presented  to 
them.  The  videotape  analysis  showed  that  this  conflict 
resolution  behavior  was  about  evenly  spread  out  over  all  three 
phases  of  mission  execution.  However,  as  a  proportion  of  the 
communications  activity  occurring  during  each  phase,  it  was  found 
that  it  comprised  most  of  the  communication  (88%)  between  Aquila 
crewmen  and  higher  headquarters  while  receiving  the  mission,  but 
only  8%  of  such  communication  during  mission  planning,  and  less 
than  1%  during  actual  flight. 

An  implication  of  these  findings  is  that  specific  training 
in  system  limitations  is  extremely  important,  especially  given 
that  searching  an  area  with  UAVl.  has  proven  to  be  a  very  slow  and 
tedious  process.  Individuals  in  higher  headquarters  who  are 
tasking  ’  AVs  may  easily  overestimate  their  search  capability  and 
it  is  crucial  for  UAV  crews  to  be  trained  to  expect  this  and  be 
prepared  to  resolve  such  problems  with  higher  headquarters. 


Manpower  and  Personnel  Selection 

The  third  topic  related  to  operation  of  the  system  focuses 
on  the  manpower  and  personnel  requirements  for  Aquila.  There  are 
two  basic  issues  here:  the  number  of  personnel  required  for 
manning  the  system,  and  the  aptitudes  required  for  performing  the 
various  tasfcs  associated  with  operating  the  system. 

Information  from  OT  II  and  an  independent  analysis  of 
manpower,  personnel  and  training  issues  (II. S.  Army  Research 
Institute,  3987'  indicated  that,  with  respect  to  the  number  of 
personnel  requi  ad,  Aquila  was  far  more  manpower  intensive  th  n 
originally  planned. 
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Operations  manpower  had  its  own  set  of  problems.  For 
example,  the  OT  II  test  criterion  for  preparing  and  launching  an 
air  vehicle  after  receiving  a  mission  order  was  a  maximum  of  60 
minutes  on  80%  of  the  trials.  However,  the  time  criterion  was 
met  on  only  44%  of  the  trials.  Part  of  the  failure  to  reach 
criterion  can  be  attributed  to  the  high  number  of  launches  that 
were  aborted  because  of  maintenance  problems  indicated  by  the 
Built-In  Test  (BIT)  system.  In  fact,  throughout  OT  II  an 
average  of  2.2  launch  attempts  were  made  for  every  successful 
launch.  Nevertheless,  the  eight  man  launch  and  recovery  section 
was  usual  y  able  to  handle  adequately  the  launch  when  equipment 
did  not  malfunction,  and  in  many  cases  got  a  launch  off  in  ten 
minutes.  However,  the  OT  II  only  tested  operations  during 
daylight  hours.  As  mentioned  earlier,  the  Army  plans  to  include 
a  FLIR  payload  on  future  UAVs  and  thus  provide  a  24-hour 
continuous  operations  capability.  Given  the  high  degree  of 
equipment  malfunction  that  kept  crews  busy,  and  the  requirement 
to  operate  around-the-clock ,  one  is  logically  led  to  ask  whether 
an  eight-man  crew  could  continue  o  function  adequately  for  very 
long.  Perhaps  two  four-man  crews  could,  by  working  12-hour 
shifts,  but  then  one  must  consider  that  many  other  duties  arise 
in  combat,  such  as  perimeter  security.  Unfortunately,  no  data 
were  collected  on  this  very  important  manpower  problem  during  any 
of  the  tests,  but  the  SRL  Task  Force  plans  to  examine  it  in 
evaluating  future  UAV  systems. 

A  related  .  lest-inn  that  emerged  ,  (and  also  one  on  which  data 
are  scarce) ,  concerned  the  skills  and  aptitudes  re  aired  for 
operating  Aquila.  A  basic  problem  existed  in  that  he  skills 
required  to  operate  a  syste:  like  Aquila  were  distinct  enough  to 
require  a  unique  military  occupational  specialty  (MOS) . 

However,  since  it  was  not  a  high  volume  system  (the  Army  only 
planned  to  acquire  nine  Aquila  batteries  requiring  about  a 
thousand  troops) ,  the  density  of  the  MOS  was  low.  This  situation 
was  complicated  by  the  fact  that  soldiers  with  the  Aquila  MOS 
were  assigned  to  one  of  two  distinct  jobs:  either  operating  the 
launch  and  recovery  system,  where  requirements  were  rather 
physical,  or  operating  the  control  station,  where  requirements 
were  more  cognitive  and  perceptual.  It  could  be  argued  that  two 
MOSs  should  have  been  created  for  these  jobs,  but  doing  so  would 
have  meant  that  each  MOS  would  contain  only  400-600  troops,  a 
very  1ow  density  indeed.  The  Army  personnel  system  has 
difficulty  managing  low  density  MOSs.  The  situation  was 
compounded  by  the  fact  that  little  empirical  data  existed  on  the 
types  of  aptitudes  actually  required  for  operating  the  control 
station,  so  it  was  difficult  to  make  a  firm  decision  as  to 
whether  or  not  that  job  should  require  a  distinct  MOS.  An 
alternative  to  the  creation  of  a  separate  MOS  is  the  designation 
of  an  additional  skill  identifier  to  differentiate  those 
personnel  whose  job  is  handling  heavy  equipment  from  those  who 
perform  primarily  cognitive  tasks.  These  are  fundamental  issues 
to  be  addressed  in  futur  ;  UAV  MANPRINT  efforts. 
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Maintainer-Related  MANPRINT  Concerns 


Estimating  Maintenance  Manpower  Re ouirements 

HARDMAN  analysis.  In  addition  to  the  operations  manpower 
problems  mentioned  in  the  preceding  section,  evidence  of 
potential  mainten  .nee  manpower  difficulties  arose  in  conjunction 
with  DT  J  .  and  OT  II.  This  phase  of  the  Aquila  MANPRINT  Task 
Force  effort  was  concerned  with  the  application  of  an  analytical 
method  for  forecasting  manpower  requirements,  and  the  impact  of 
poor  performance  of  the  electronic  fault  isolation  equipment 
during  OT  II  on  the  accuracy  of  manpower  estimates. 

Hardware  vs.  manpower  (HARDMAN)  and  its  automated 
derivative,  HARDMAN  II  are  methods  for  estimating  MPT 
requirements  for  emerging  systems.  HARDMAN  was  performed  on  the 
Lockheed  Aquila  RPV  for  the  Field  Artillery  School  TRADOC  Systems 
Manager  under  contract  with  Jet  Propulsion  Laboratory  (Dynamics 
Research  Corporation,  1983).  The  original  analysis  assumed  that 
the  RPV  battery  would  be  fielded  in  five  autonomous  sections,  and 
that  the  operational  scenario  would  be  a  12-hour  day. 

The  1985  revision  of  the  Target  Acquisition,  Designation 
and  Aerial  Reconnaissance  System  (TADARS)  RPV  Operational  and 
Organizational  (O&O)  Plan  imparted  more  centralization  and 
control  to  the  Aquila  battery.  Originally,  the  battery  was  to 
consist  of  five  autonomous  sections,  each  with  the  full 
capability  of  launching,  piloting,  recovering  and  maintaining  a 
total  of  five  air  vehicles  (AVs) ;  in  short,  a  section  could 
conduct  a  complete  mission  on  its  own.  This  arrangement  was 
superseded  by  a  more  centralized  battery,  consisting  also  of  five 
sections,  none  of  which  was  fully  autonomous.  The  Aquila  battery 
now  consisted  of  three  Forward  Control  Sections  (FCSs)  and  two 
rear  area  Centralized  Launch  and  Recovery  Sections,  (CLRSs)  each 
with  its  own  Ground  Control  Station.  Each  CLRS  would  have  a 
Launch  Subsystem,  Recovery  Subsystem,  Air  Vehicle  Handler  and 
five  AVs.  It  was  the  responsibility  of  the  two  CLR; s  to  conduct 
launch  and  recovery  operations  for  the  entire  battery.  After  the 
AV  was  launched,  it  was  to  be  handed  oft  to  a  FCS,  which 
controlled  the  aircraft  as  it  carried  out  its  mission.  The  AV 
was  to  be  handed  off  to  a  CLRS  for  recovery  upon  completion  of 
its  mission.  Only  one  CLRS,  the  Primary  CLRS  (or  CLRS  l)  was  to 
have  a  maintenance  shelter  (MS)  along  with  three  spare  AVs.  The 
MS  and  its  crew  of  four  were  to  provide  organizational  level 
maintenance  for  all  13  AVs.  Originally,  it  was  intended  that 
each  CLRS  have  a  MS.  Figure  1  illustrates  the  employment  of  the 
Aquila  battery  under  the  CLRS  0&0  Plan. 
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Figure  1.  Aquila  battery  employment. 


Analysis  of  the  original  HARDMAN  was  required  to  incorporate 
these  changes.  One  principal  finding  of  the  revised  HARDMAN 
(Dynamics  Research  Corporation,  1985)  was  that,  because  of  the 
shift  from  a  12  t'  a  24  hour  operational  scenario,  six  MOS  13T  P9 
maintainers  were  required  at  the  MS  instead  of  the  four  required 
by  the  1985  TADARS  O&O  Plan. 

Sensitivity  analysis.  However,  lower  than  expected 
automatic  fault  isolation  (FI)  rates  obtained  by  the  automated 
test  equipi  ent  (ATE)  during  DT  II  in  19  6  and  CT  II  in  1987 
indicated  tiat  maintenance  manpower  requirements  should  be 
readdr'  ;sed.  Also,  it  appeared  that  single  point  estimates  of 
ATE  performance  were  not  as  useful  as  a  range  of  estimates  based 
on  expected  levels  of  ATE  performance. 

In  an  investigation  of  another  Army  system,  Stewart  and 
Shvern  (1988)  applied  HARDMAN  II  sensitivity  analysis  to  two 
components  of  the  Forward  Area  Air  Defense  system.  Maintenance 
manpower  estimates  as  a  function  of  automatic  fault  isolation 
performance  caused  the  Army  to  reexamine  its  original  maintenance 
nu  npower  requirements.  Similarly,  the  Aquila  sensitivity 
analysis  was  performed  in  order  to  provide  the  Army  with  updated 
information  about  maintenance  requirements  for  the  Aquila  RPV, 
and  to  illustrate  further  its  efficacy  as  an  adjunct  to  HARDMAN 
and  its  derivatives. 


Method 

Documentation.  The  principal  dat  a  sources  were  the  1983  and 
1985  HARDMAN  analyses.  The  1985  TADA  S  0&0  Plan  (with  changes) 
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provided  information  on  projected  repair  times,  usage  rates,  and 
operational  scenarios.  Test  data  were  available  from  DT  II  and 
OT  II  which  provided  information  on  maintenance  ratios,  the 
number  of  repair  actions  at  organizational  and  int  ^rmediate 
levels,  and  operational  availability  estimates  frc  .1  these  tests. 

The  Required  Operational  Capabilities  (TOC)  document  called 
for  a  successful  fault  isolation  (FI)  rate  of  90%.  Generally, 

ATE  performance  to  date  has  been  poorer  than  this  (see  Nauta, 
1985).  During  Development  Test  II  (Cozby,  1986),  the  Aquila  ATE 
system  only  isolated  35%  of  all  faults.  During  Operational  Test 
II  (OTEA,  1987)  this  rate  was  slightly  less  than  20%. 

Analytical  approach.  The  methodology  employed  in  ♦'he 
present  analysis  was  a  "top-down"  approach  which  relied  on 
HARDMAN  results  as  a  baseline.  Because  detailed  raw  data  from 
the  HARDMAN  were  not  available,  it  was  necessary  to  rely  on  data 
from  DT  II  and  OT  II  to  obtain  estimates  of  AV  down  time,  repair 
times  and  maintenance  ratios.  The  wartime  operational  scenario 
from  the  1985  TADARS  0S.0  Plan  allowed  for  extrapolation  to  the 
total  RPV  battery.  The  resultant  annual  maintenance  manhours 
(AMMH)  obtained  through  the  top-down  approach  (7722)  agreed 
closely  with  those  from  the  HARDMAN  (7961  adjusting  for  the  24 
hour  scenario) .  Mean  Time  to  Repair  (MTTR)  times  using  ATE  from 
the  revised  O&O  Plan  are  wrench-turning  times  only.  Using  the 
ATE,  these  10  min  by  day  and  45  min  by  night.  Manual  MTTRs 

were  90  min  (day)  and  135  min  (night) . 

The  ATE  system  is  mounted  inside  the  MS.  For  diagnoses  of 
faults  to  be  carried  out,  the  AV  must  be  partially  disassembled, 
defueled,  and  then  moved  inside  the  shelter,  accounting  for  the 
day-night  differences  in  MTTR. 

Operational  scenarios.  Total  AV  operating  hours  for  the 
24  hour-25  mission  scenario  would  be  54  hours,  based  on  the  1935 
TADARS  O&O  Plan. 

Maintenance  ratios.  From  operational  requirements  and  OT 
II  test  results  it  can  be  inferred  that  1.27  maintenance  actions 
per  day  will  be  required  per  air  vehicle. 

Effects  of  travel.  For  the  remote  CLRS,  there  should  be 
6.35  maintenance  actions  per  day  anticipated.  It  is  assumed  that 
two  maintainers  from  the  maintenance  shelter  will  retrieve  the 
AV. 


Results  and  Discussion 

The  sensitivity  estimates  are  presented  in  Table  1. 

Dependent  variables  are  annual  maintenance  manhours  (AMMH) ,  final 
maintenance  ratio  (FMR) ,  operational  availability  (A0) ,  number  of 
mission-ready  AVs  and  maintenance  man  years  (MMY) .  Independent 
variables  are  (a)  ATE  fault  isolation  (FI)  rates  of  90%  (similar 
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to  the  ROC  requirement) ,  40%  (slightly  better  than  at  DT  II)  and 
20%  (similar  to  OT  II) ,  (b)  percentage  of  repairs  performed 
during  daytime  hours  (  50%  or  80%)  and  (c)  distance  between  CURS 
1  and  2.  Distance  between  CLRS=0  is  equivalent  to  there  being 
two  MSs. 


Table  I 

Aquila  RPV  Maintenance  Sensitivity  Analysis 


Distance  Between  Primary  and 

Secondary  CURS 

(Km) 

co 

VO 

o 

10  12 

14 

ATE  FI 

-  9.0% 

(50% 

daytime  repairs) 

AMKH 

7961 

8579 

8888 

9197 

9506 

9815 

10124 

FMR 

.40 

.44 

.45 

.47 

.48 

.50 

.51 

Ao 

.60 

.56 

.  54 

.53 

.52 

.50 

.49 

AVs 

7.75 

7.34 

7.13 

6.93 

6.72 

6.52 

6.32 

MKi. 

3.32 

3.57 

3.70 

3 . 83 

3.96 

4,08 

4 . 22 

(80% 

daytime  repairs) 

AMMH 

5998 

6464 

6697 

6938 

7188 

7447 

7715 

FMR 

.30 

.33 

.34 

.35 

.  36 

.38 

.39 

Aq 

.70 

.66 

.66 

.65 

.64 

.62 

.61 

AVs 

9.10 

8.50 

8.58 

8.45 

8.28 

8.06 

7.93 

MMY 

2.50 

2.69 

2 . 69 

2 .89 

2.98 

3 . 10 

3.21 

ATE  FI 

-  40% 

(50%  daytime 

repairs) 

AMMH 

13988 

14605 

14915 

152  2  4 

15533 

15824 

16151 

FMR 

.71 

.74 

.76 

.77 

.79 

.80 

.82 

A0 

.29 

.26 

.24 

.23 

.  22 

.20 

.18 

AVs 

3.77 

3.38 

3 . 12 

2.99 

2 . 86 

2.60 

2.34 

MMY 

5.82 

6.08 

6.21 

6.34 

6.47 

6.60 

6.72 
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Table  1  (Continued) 

Acjuila  RPV  Maintenan  a  Sensitivity  Analysis 


Distance  Between  Primary 

and  Secondary  CLRS  (Km) 

0  4  6  8 

10  12  14 

ATE  FI 

=  40% 

(80%  daytime 

repairs) 

AMMH 

10966 

11818 

12243 

12684 

13141 

13614 

14104 

FMR 

.56 

.  60 

.62 

.  64 

.67 

.  69 

.72 

.44 

.40 

.38 

.  36 

.33 

.31 

.  28 

AVs 

5.72 

5.20 

4.94 

4 . 68 

4 . 29 

4 . 03 

3 . 64 

MMY 

4 . 57 

4 .92 

5.10 

5.29 

5.48 

5.67 

5.88 

ATE  FI 

=  20% 

(50%  < 

daytime 

repairs) 

*  vjtvtj 

Ai’iiui 

16398 

17016 

17325 

17634 

17  9  4  2 

18252 

i  h  6  e>  j. 

FMR 

.83 

.86 

.88 

.90 

.91 

.S3 

.94 

.17 

.  14 

.12 

.10 

.  09 

.07 

.06 

AVs 

2 . 18 

1.82 

1.56 

1.30 

1.17 

.91 

.78 

MMY 

6.83 

7.09 

7.22 

7.35 

7.48 

7.61 

7.73 

(80% 

daytime 

repairs) 

AMMH 

12963 

13969 

14993 

15201 

15534 

16093 

16672 

FMR 

.  66 

.71 

.77 

.76 

.79 

.82 

.85 

Ao 

.  34 

.29 

.24 

.23 

.21 

.  IS 

.15 

AVs 

4 . 42 

3.77 

3.33 

2.99 

2.73 

2.34 

1.95 

!  '.MY 

5.40 

5.82 

6.25 

6.33 

6.47 

6.71 

6.95 

Conclusions 


Maintenance  support  of  the  Aquila  would  nave  posed  no 
problem  if  the  ATE  system  were  to  have  performed  as  specified  in 
the  06.0  Plan.  The  sensitivity  analyses  showed  that  for  ATE  FI 
rates  of  90%,  the  maximum  mission  requirement  (of  five  AVs 
airborne  at  any  one  time)  could  have  been  met  with  the 
maintenance  manpower  resources  available.  Neither  repair 
scheduling,  the  number  of  MSs,  nor  distance  between  CLRS  would 
have  posed  a  threat  to  operational  capability. 
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Were  fl  rates  to  fall  to  40%,  the  five-AV  requirement  could 
still  be  met  under  optimal  conditions  (80%  daytime  repairs  and 
two  MSs)  ;  with  only  one  MS,  the  CLRSs  must  be  no  -farther  than  4 
km  apart  in  order*  to  meet  the  requirement. 

If  FI  rates  approximated  OT  II  results  (20%) ,  the  mission 
requirement  could  not  be  met,  regardless  of  assets  or  scheduling 
of  repair  times. 

It  should  be  noted  that  estimates  for  the  present  project 
agree  closely  with  findings  of  the  draft  Human  Factors 
Engineering  Analysis  (Human  Engineering  Laboratory,  1987)  which 
found  that  during  OT  II  the  MS  crew  of  a  battery  "minus"  had 
difficulty  keeping  up  with  the  workload  for  a  single  CLRS .  The 
report  also  expressed  doubt  that  the  MS  crew  would  be  able  to 
support  an  entire  Aquila  battery.  If  operational  requirements  of 
the  Aquila  are  to  be  met  at  all,  an  FI  rate  of  at  least  40%, 
which  is  slightly  higher  than  that  attained  at  FT  II,  must  be 
achieved,  along  with  the  acquisition  of  an  additional  MS  for  each 
battery. 

The  present  analysis  further  underscores  the  usefulness  of 
sensitivity  analysis,  which  enhances  the  effectiveness  of  HARDMAN 
II.  The  uncertain  performance  of  ATE  and  other  electronic  fault 
diagnostics  make  single  point  estimates,  based  on  optimistic 
uiik-cria ,  impractical . 


Genera]  Discussion 


Lessons  Learned 

Succinctly,  it  can  be  stated  that  the  Aquila  MANPRINT  Task 
Force's  intervention  came  too  late  to  save  a  foundering  system 
from  cancellation.  In  spite  of  this,  major  MANPRINT  problems 
relating  to  the  operation  and  maintenance  of  Aquila  were 
pinpointed-  painstakingly  analyzed,  and  concrete  recommendations 
were  made  which  should  provide  valuable  lessons  for  those 
involved  in  the  development  of  the  current  family  of  UAVs.  These 
recommendations  illustrated  that  some  MPT  problems  can  be 
alleviated  or  at  least  minimized  without  requiring  expensive 
redesign  of  the  system.  In  fact,  with  the  exception  of  the 
imaging  system  it  itself,  the  difficulties  encountered  appeared 
to  be  attributable  more  to  MPT  than  to  inherent  hardware  design 
problems. 

In  addition,  the  effectiveness  of  sensitivity  analysis  as  a 
useful  adjunct  to  HARDMAN  and  HARDMAN  II  was  demonstrated.  One 
important  lesson  that  carried  over  from  applications  of  the 
technique  to  other  systems  (Stewart  &  Shvern,  1988)  was  that 
performance  specifications  for  BIT  and  ATE  equipment,  at  least 
with  regard  to  many  new  and  recently-fielded  Army  systems,  may  be 
too  rigourous.  Consequently,  the  results  of  HARDMAN  II  analyses 
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that  assume  high  levels  of  BIT  and  ATE  performance  become  suspect 
when  DT  and  QT  results  show  much  lower  levels.  This  is  a  cogent 
argument  for  providing  a  range  of  manpower  estimates  based  upon  a 
band  of  BIT  or  ATE  isolation  rates.  In  such  a  way  decision¬ 
makers  can  determine  the  minim  lly  acceptable  level  of 
performance  that  would  allow  the  system  to  be  field  within  the 
stipulated  manpower  constraints  that  are  now  imposed  by  TRADOC. 


In  Retrospect 

In  brief,  what  the  ARI  MANPRINT  Task  Force  has  learned  from 
its  experience  with  Aquila  is  icself  a  good  argument  for 
MANPRINT.  The  Aquila  had  a  protracted  de%relopmental  history 
during  which  many  MPT  and  some  system  design  problems  were 
encountered.  One  major  requirement  for  successful  MANPRINT 
intervention  is  timeliness,  and  it  appears  that  for  Aquila,  the 
MPT  problems  could  and  should  have  been  addressed  much  earlier. 
Had  they  been,  the  fate  of  Aquila  may  have  been  quite  different. 
Still,  it  is  reasonable  to  suppose  that  similar  problems  will  be 
encountered  with  the  UAV  Close  system,  Aquila's  successor,  which 
is  currently  under  development.  Now  is  the  time  to  identify, 
attack,  and  resolve  these  problems. 
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